Virtual trauma patient simulation design using the McGill Simulation Complexity Score (MSCS): a breakthrough in trauma education.

BACKGROUND: Virtual patient simulations are interactive, computer-based cases. We designed scenarios based on the McGill Simulation Complexity Score (MSCS), a previously described objective complexity score. We aimed to establish validity of the MSCS and introduce a novel learning tool in trauma education at our institution. METHODS: After design of an easy and difficult patient scenario, we randomized medical students and residents to each perform 1 of the 2 scenarios. We conducted a 2-way analysis of variance of training level (medical student, resident) and scenario complexity (easy, difficult) to assess their effects on virtual time, the number of steps taken in the scenario, beneficial and harmful actions, and the ratio of beneficial over harmful actions. RESULTS: Virtual patient scenarios were successfully designed using the MSCS. Twenty-four medical students and 12 residents participated in the easy scenario (MSCS = 3), and 27 medical students and 12 residents did the difficult scenario (MSCS = 18). Though beneficial actions were similar between students and residents, sudents performed more harmful actions, particularly when the scenario was difficult. One virtual patient died in the easy scenario and 3 died in the difficult one (all medical students). Performance varied with level of complexity and there was significant interaction between level of training and number of steps, as well as with number of harmful actions. Decreasing performance with increasing level of complexity, as defined by the MSCS, suggests this score can accurately quantify difficulty. CONCLUSION: We established validity of the MSCS and showed its successful application on virtual patient scenario design.

Development of an evidence-based educational resource in oncology: 'Living safely with bone metastases'.

Objective: To create an evidence-based patient education resource to better support cancer patients with bone metastases in carrying out safe movements during activities of daily living, to maintain their bone health and reduce the risk of fractures. Methods: A quality improvement project was conducted in three phases: Development of the Resource, Preliminary Feedback and Revision, and French Canadian Translation. Results: The educational resource Living Safely with Bone Metastases focuses on safe movement, activities of daily living, and exercise, organized within the sections Move with care, Stay safe in different environments and Follow an exercise program prescribed by a physiotherapist. Translation yielded a Canadian French version Vivre en toute sécurité avec des métastases osseuses. Conclusion: Living Safely with Bone Metastases is an accessible online and paper resource for patients and healthcare professionals, in order to promote ongoing disease management of individuals with bone metastases. Innovation: Cancer patients with bone metastases are at high risk of pathological fractures however resources on fracture prevention are lacking. Living Safely with Bone Metastases is an innovative health education resource that fills an important gap in oncology practice and has the potential to reduce the occurrence of fractures.

Objective Assessment of the Entrustable Professional Activity Handover in Undergraduate and Postgraduate Surgical Learners.

OBJECTIVE: This study used a virtual patient simulation (VPS) to quantifiably and objectively assess undergraduate (UG) to postgraduate (PG) medical learners' acquisition of the entrustable professional activity (EPA) "handover," focusing particularly on the transition to residency. This EPA is critical because it is part of a core competency for UG and PG training in both the United States and Canada, and is essential for patient safety and comprehensive professional communication. DESIGN: Data were collected from 3 separate groups of participants: 2 UG cohorts from an earlier study, as well as a PG cohort at the beginning of residency. All participants completed the same trauma VPS, which required a free text summary statement that was used as a surrogate for an oral handover. These were collected and scored independently, using previously developed validated rubrics, one procedural and the second semantic. SETTING: All study participants were from one site. The VPS case was completed online. PARTICIPANTS: Two different UG groups, one designated junior (N = 52), was studied at the beginning of their clerkship year, a second group, designated senior (N = 30), was studied at the end of their clerkship year. These groups were compared to a third group of PG learners (N = 31) during the initial 2 weeks of their residency. Informed consent was obtained from all participants. RESULTS: A procedural rubric assessed learners' cognitive knowledge of trauma care-management. A semantic rubric assessed their use of the professional language necessary for a safe and succinct clinical handover communication. An Analysis of Variance comparing scores on the procedural rubric was highly significant with Tukey LSD tests indicating that all 3 groups were significantly different. Students increased their scores on the procedural rubric at each stage of their training. A parallel Analysis of Variance comparing students' scores on the semantic rubric revealed no significant increase in scores, indicating that students did not improve in their capacity to communicate professionally as they progressed through their training. CONCLUSIONS: Taken together, these results demonstrate that training was successful in teaching cognitive-based procedures, but not effective in teaching professional communication, which is critical to the EPA handover. Greater emphasis needs to be placed on ensuring the acquisition of professional communication skills throughout the continuum of UG and PG clinical activities. Faculty development should serve as a support to assist medical educators to address this requirement. These results also demonstrate that VPS with associated objective and validated rubrics can be used as an assessment methodology to quantifiably measure learner performance with respect to the EPA handover. A similar strategy should be considered across the UG and PG continuum for other EPAs and could form the nexus for further research.

Creation and Pilot-testing of Virtual Patients for Learning Oncologic Emergency Management.

Purpose or objective Management of oncologic emergencies becomes critical at the start of the second year of a radiation oncology residency. Considering the limited exposure to oncology in the medical school curriculum, this knowledge gap needs to be filled prior to managing real patients. The aim of this project was to create virtual patients (VPs) to ease this transition and improve learner readiness for independently managing oncologic emergencies on call. Material and methods A curriculum mapping exercise was done to identify gaps. The main oncologic emergencies that needed to be addressed were selected for development of the modules. Review of the key concepts for management was elucidated and validated. These included history, physical examination, imaging interpretation, staging, as well as anatomy, epidemiology, pertinent literature, differential diagnosis, prognostication, radiation treatment planning, summarizing, and patient- and peer-communication skills. Clinical vignettes were then designed, in collaboration with a virtual patient education expert, to mimic the clinical presentation and evolution of a typical patient for three common oncologic emergencies: spinal cord compression, superior vena cava syndrome, and tumor-induced hemorrhage. Results Three virtual modules were developed: spinal cord compression, superior vena cava syndrome, and tumor-induced hemorrhage. Each case included 25 to 30 vignettes that participants progressed through, with a total estimated completion time of 30 to 45 minutes. Each node branched out to provide a detailed answer and explanation of the key concept. Figures were included to mimic real patients and to provide a more authentic learning experience. The modules also included quantitative pre- and post-testing assessments, including multiple-choice questions, true or false, fill in the blank, short answers, and text response. The cases were then transcribed onto a virtual patient simulation platform. Following completion of the module, a report was generated for each individual learner to track all responses and used as the assessment tool. The pilot test showed an increase of 28% in the pre-to-post-test results in a cohort of nine residents. The mean pre-test result of 58% increased to a mean post-test result of 86% (range: 70-100%) after completing the three modules. Conclusion VPs can be used for learning the management of oncologic emergencies and can be done on a simulation-based learning platform. The modules can be used as both, a learning and an assessment tool for junior residents. The results of the pilot test show a significant improvement in knowledge acquisition between pre- and post-test scores after completion of the three modules.

Development and Validation of a Tool to Evaluate the Evolution of Clinical Reasoning in Trauma Using Virtual Patients.

CONTEXT: Undergraduate medical students at a large academic trauma center are required to manage a series of online virtual trauma patients as a mandatory exercise during their surgical rotation. PURPOSE: Clinical reasoning during undergraduate medical education can be difficult to assess. The purpose of the study was to determine whether we could use components of the students' virtual patient management to measure changes in their clinical reasoning over the course of the clerkship year. In order to accomplish this, we decided to determine if the use of scoring rubrics could change the traditional subjective assessment to a more objective evaluation. BASIC PROCEDURES: Two groups of students, one at the beginning of clerkship (Juniors) and one at the end of clerkship (Seniors), were chosen. Each group was given the same virtual patient case, a clinical scenario based on the Advanced Trauma Life Support (ATLS) Primary Trauma Survey, which had to be completed during their trauma rotation. The learner was required to make several key patient management choices based on their clinical reasoning, which would take them along different routes through the case. At the end of the case they had to create a summary report akin to sign-off. These summaries were graded independently by two domain "Experts" using a traditional subjective surgical approach to assessment and by two "Non-Experts" using two internally validated scoring rubrics. One rubric assessed procedural or domain knowledge (Procedural Rubric), while the other rubric highlighted semantic qualifiers (Semantic Rubric). Each of the rubrics was designed to reflect established components of clinical reasoning. Student's t-tests were used to compare the rubric scores for the two groups and Cohen's d was used to determine effect size. Kendall's τ was used to compare the difference between the two groups based on the "Expert's" subjective assessment. Inter-rater reliability (IRR) was determined using Cronbach's alpha. MAIN FINDINGS: The Seniors did better than the Juniors with respect to "Procedural" issues but not for "Semantic" issues using the rubrics as assessed by the "Non-Experts". The average Procedural rubric score for the Senior group was 59% ± 13% while for the junior group, it was 51% ± 12% (t(80)= 2.715; p = 0.008; Cohen's d = 1.53). The average Semantic rubric score for the Senior group was 31% ± 15% while for the Junior group, it was 28% ± 14% (t(80) = 1.010; p = .316, ns). There was no statistical difference in the marks given to the Senior versus Junior groups by the "Experts" (Kendall's τ = 0.182, p = 0.07). The IRR between the "Non-Experts" using the rubrics was higher than the IRR of the "Experts" using the traditional surgical approach to assessment. The Cronbach's alpha for the Procedural and Semantic rubrics was 0.94 and 0.97, respectively, indicating very high IRR. The correlation between the Procedural rubric scores and "Experts" assessment was approximately r = 0.78, and that between the Semantic rubric and the "Experts" assessment was roughly r = 0.66, indicating high concurrent validity for the Procedural rubric and moderately high validity for the Semantic rubric. PRINCIPLE CONCLUSION: Clinical reasoning, as measured by some of its "procedural" features, improves over the course of the clerkship year. Rubrics can be created to objectively assess the summary statement of an online interactive trauma VP for "procedural" issues but not for "semantic" issues. Using IRR as a measure, the quality of assessment is improved using the rubrics. The "Procedural" rubric appears to measure changes in clinical reasoning over the course of 3rd-year undergraduate clinical studies.

Identifying high quality medical education websites in Otolaryngology: a guide for medical students and residents.

BACKGROUND: Learners often utilize online resources to supplement formalized curricula, and to appropriately support learning, these resources should be of high quality. Thus, the objectives of this study are to develop and provide validity evidence supporting an assessment tool designed to assess the quality of educational websites in Otolaryngology- Head & Neck Surgery (ORL-HNS), and identify those that could support effective web-based learning. METHODS: After a literature review, the Modified Education in Otolaryngology Website (MEOW) assessment tool was designed by a panel of experts based on a previously validated website assessment tool. A search strategy using a Google-based search engine was used subsequently to identify websites. Those that were free of charge and in English were included. Websites were coded for whether their content targeted medical students or residents. Using the MEOW assessment tool, two independent raters scored the websites. Inter-rater and intra-rater reliability were evaluated, and scores were compared to recommendations from a content expert. RESULTS: The MEOW assessment tool included a total of 20 items divided in 8 categories related to authorship, frequency of revision, content accuracy, interactivity, visual presentation, navigability, speed and recommended hyperlinks. A total of 43 out of 334 websites identified by the search met inclusion criteria. The scores generated by our tool appeared to differentiate higher quality websites from lower quality ones: websites that the expert "would recommend" scored 38.4 (out of 56; CI [34.4-42.4]) and "would not recommend" 27.0 (CI [23.2-30.9]). Inter-rater and intra-rater intraclass correlation coefficient were greater than 0.7. CONCLUSIONS: Using the MEOW assessment tool, high quality ORL-HNS educational websites were identified.

A Learner-Created Virtual Patient Curriculum for Surgical Residents: Successes and Failures.

OBJECTIVE: To determine the feasibility and effectiveness of a learner-created virtual patient (VP) curriculum for postgraduate year 2 surgical residents. DESIGN: Using a social-constructivist model of learning, we designed a learner-created VP curriculum to help postgraduate year 2 residents prepare for their in-training surgical examination. Each resident was assigned to create a VP curriculum based on the learning objectives for this examination, and VP cases were then disseminated to all residents for completion. To measure the learning effects of the curriculum, participants completed 2 simulated in-training examinations, both at the beginning and at the end of the intervention. Study participants also participated in a focus group and completed an online questionnaire about the perceived learning value of the curriculum. SETTING: The study was conducted at the McGill University Health Centre, a tertiary care hospital in Montreal, Canada. PARTICIPANTS: In total, 24 residents from 7 surgical specialties completed both the pretest and posttest, as well as took part in the creation of a VP curriculum. Of those 24 residents, only 19 residents completed the cases created by their peers, with 7 completing greater than 50% of the cases and 12 completing less than 50%. In all 17 residents responded to the online questionnaire and 11 residents participated in the focus group. RESULTS: The VP curriculum failed to improve scores from pretest (59.6%, standard deviation = 8.1) to posttest (55.4%, standard deviation = 6.6; p = 0.01) on the simulated in-training examination. Nonetheless, survey results demonstrated that most residents felt that creating a VP case (89%) and completing cases created by their peers (71%) had educational value. Overall, 71% preferred active participation in a curriculum to traditional didactic teaching. The focus group identified time-related constraints, concern about the quality of the peer-created cases, and questioning of the relationship between the curriculum and the Surgical Foundations examination as barriers to the success of the curriculum. CONCLUSIONS: Despite the fact that a learner-created VP curriculum did not improve scores on a mock in training examination, residents viewed this intervention as a valuable educational experience. Although there were barriers to the implementation of a learner-created curriculum, it is nonetheless important to try and integrate pedagogical concepts into the instructional design of curricula for surgical residents.

Twelve tips to support the development of clinical reasoning skills using virtual patient cases.

Clinical reasoning is a critical core competency in medical education. Strategies to support the development of clinical reasoning skills have focused on methodologies used in traditional settings, including lectures, small groups, activities within Simulation Centers and the clinical arena. However, the evolving role and growing utilization of virtual patients (VPs) in undergraduate medical education; as well as an increased emphasis on blended learning, multi-modal models that include VPs in core curricula; suggest a growing requirement for strategies or guidelines that directly focus on VPs. The authors have developed 12 practical tips that can be used in VP cases to support the development of clinical reasoning. These are based on teaching strategies and principles of instructional design and pedagogy, already used to teach and assess clinical reasoning in other settings. Their application within VPs will support educators who author or use VP cases that promote the development of clinical reasoning.

Improving undergraduate medical education about pain assessment and management: a qualitative descriptive study of stakeholders' perceptions.

BACKGROUND: Pain is one of the most common reasons for individuals to seek medical advice, yet it remains poorly managed. One of the main reasons that poor pain management persists is the lack of adequate knowledge and skills of practicing clinicians, which stems from a perceived lack of pain education during the training of undergraduate medical students. OBJECTIVE: To identify gaps in knowledge with respect to pain management as perceived by students, patients and educators. METHODS: A qualitative descriptive study was conducted. Data were generated through six focus groups with second- and fourth-year medical students, four focus groups with patients and individual semistructured interviews with nine educators. All interviews were audiotaped and an inductive thematic analysis was performed. RESULTS: A total of 70 individuals participated in the present study. Five main themes were identified: assessment of physical and psychosocial aspects of pain; clinical management of pain with pharmacology and alternative therapies; communication and the development of a good therapeutic relationship; ethical considerations surrounding pain; and institutional context of medical education about pain. CONCLUSION: Participating patients, students and pain experts recognized a need for additional medical education about pain assessment and management. Educational approaches need to teach students to gather appropriate information about pain, to acquire knowledge of a broad spectrum of therapeutic options, to develop a mutual, trusting relationship with patients and to become aware of their own biases and prejudice toward patients with pain. The results of the present study should be used to develop and enhance existing pain curricula content.

Developing an interprofessional care plan for an older adult woman with breast cancer: from multiple voices to a shared vision.

Interprofessional collaboration is central to quality patient care; however, little is known about developing interprofessional care plans, particularly in oncology. This article describes the development of an interprofessional care plan for an older adult woman with breast cancer. Two collaborative expert workshops were used; 15 clinical experts reviewed an online patient case and were asked to prepare a uniprofessional care plan. In workshop 1, participants worked from a draft interprofessional care plan, synthesized from the uniprofessional care plans by research associates, to arrive at consensus on an ideal interprofessional care plan. Using qualitative inductive content analysis of workshop transcripts, specific changes and overall key principles were identified and used to revise the draft plan. Based on these findings, a generalized interprofessional care plan/oncology model was developed. Revisions and proposed model were validated through consensus by participants during workshop 2. Participants highlighted the iterative, cyclical, and multilayered nature of patient care experiences; the importance of central patient profiles, which are contributed to and validated by all healthcare professionals; and the importance of assessing patient understanding. Participation of a patient representative provided an invaluable contribution. The process and model provide a unique framework for interprofessional care plan development in other settings and patient populations.

Learning and caring in communities of practice: using relationships and collective learning to improve primary care for patients with multimorbidity.

We introduce a primary care practice model for caring for patients with multimorbidity. Primary care for these patients requires flexibility and ongoing coordination, and it often must be tailored to individual circumstances. Such complex and flexible care could be accomplished within communities of practice, whose participants are willing to learn from their shared practice, further each other's goals, share their stories of success and failure, and promote the continued evolution of collective learning. Primary care in these communities would be conceived as a complex adaptive process in which the participants use an iterative approach to care improvement that integrates what they learn and do collectively over time. Clinicians in these communities would define common goals, cocreate care plans, and engage in reflective case-based learning. As community members manage their knowledge, gain insights, and develop new care strategies, they can improve care for patients with multiple conditions. Using a mix of methods, future research should explore the conditions that are necessary for collective learning within communities of clinicians who care for patients with multimorbidity and who develop new knowledge in practice. By understanding these conditions, we can foster the development of collective learning and improve primary care for these patients.

12 Tips: Guidelines for authoring virtual patient cases.

BACKGROUND: Virtual patient cases are an increasingly utilized and compelling pedagogical strategy for medical education informatics. They provide educators with the opportunity to develop richly layered, multidimensional teaching situations for their learners. However, 'virtual patients are notoriously difficult to author, adapt and exchange' (MedBiquitous Virtual Patient Specification, Virtual Patient Working Group 2007), and case creation can be daunting. Authors may be uncertain about the process of virtual patient case development and this can translate into ambiguity and hesitation. AIMS: This installment of the '12 tips' presents specific guidelines that are intended to provide medical educators with guidelines to facilitate the development of virtual patient cases. METHODS: These 12 tips are based upon comprehensive, research-based, theory-grounded and criterion-referenced guidelines and founded in pedagogical principles, theories of cognition, and recognition of current technology and availability of authoring applications. RESULTS: It is anticipated that the 12 tips will provide medical educators interested in authoring virtual patient cases one set of useful guidelines to facilitate the process. CONCLUSIONS: Virtual patient cases provide medical educators with an innovative tool for medical education. These guidelines will assist authors in case development.

Learning while having fun: the use of video gaming to teach geriatric house calls to medical students.

Although most health professionals perform home visits, there is not a structured method for performing them. In addition, in-training health professionals' exposure to home visits is limited for logistical reasons. A new method for medical students to learn how to perform an effective home visit was developed using an instructional video game. It was expected that students would learn the principles of a home visit using a video game while identifying the usefulness of video gaming (edutainment) in geriatrics education. A video game was created simulating a patient's house that the students were able to explore. Students played against time and distracters while being expected to click on those elements that they considered to be risk factors for falls or harmful for the patient. At the end of the game, the students received feedback on the chosen elements that were right or wrong. Finally, evaluation of the tool was obtained using pre- and posttests and pre- and postexposure feedback surveys. Fifty-six fourth-year medical students used the video game and completed the tests and the feedback surveys. This method showed a high level of engagement that is associated with improvement in knowledge. Additionally, users' feedback indicated that it was an innovative approach to the teaching of health sciences. In summary, this method provides medical students with a fun and structured experience that has an effect not only on their learning, but also on their understanding of the particular needs of the elderly population.

From the facts to the screen: a blended model of teaching basic hospital skills to 2nd year medical students.

We report an interactive course design which simulated the clinical setting for practicing basic hospital skills. The two-part course included (1) instructor-led, Flash-based interactive lectures; and (2) web-based case using interactive hospital forms. Our program evaluation showed that integrated sessions complemented with web-based interactive cases and hospital forms offered a better student preparation for the Geriatric Medicine clerkship and improved tutor perceptions of student performance during early rotations in the academic year.

New directions in medical e-curricula and the use of digital repositories.

Medical educators involved in the growth of multimedia-enhanced e-curricula are increasingly aware of the need for digital repositories to catalogue, store and ensure access to learning objects that are integrated within their online material. The experience at the Faculty of Medicine at McGill University during initial development of a mainstream electronic curriculum reflects this growing recognition that repositories can facilitate the development of a more comprehensive as well as effective electronic curricula. Also, digital repositories can help to ensure efficient utilization of resources through the use, re-use, and reprocessing of multimedia learning, addressing the potential for collaboration among repositories and increasing available material exponentially. The authors review different approaches to the development of a digital repository application, as well as global and specific issues that should be examined in the initial requirements definition and development phase, to ensure current initiatives meet long-term requirements. Often, decisions regarding creation of e-curricula and associated digital repositories are left to interested faculty and their individual development teams. However, the development of an e-curricula and digital repository is not predominantly a technical exercise, but rather one that affects global pedagogical strategies and curricular content and involves a commitment of large-scale resources. Outcomes of these decisions can have long-term consequences and as such, should involve faculty at the highest levels including the dean.

Development of an undergraduate medical curriculum: the McGill experience.

In 1997 the Faculty of Medicine at McGill University received a grant from the Molson Foundation. The primary project deliverable, which the authors describe, was an online, multimedia-enhanced, undergraduate medical curriculum. The decision to develop an electronic curriculum was predicated on the belief that the integration of educational technology within mainstream material delivered a "value added" component to both students and faculty, which would, in turn, facilitate teaching and learning. Pedagogical values were deemed to include: (1) the ability to use the media to implement adult learning principles such as learner-centered, self-directed and guided learning, (2) the inherent interactivity of the technology, (3) the potential of the technology to provide a powerful means for fostering forms of "termless" learning that students will need to practice medicine, (4) recognition that use of multimedia can address, in part, the variety of learning styles evidenced by students in the lecture hall and classroom, and (5) the provision of opportunities for horizontal and vertical curricular integration. In addition, it was anticipated that an electronic curriculum would permit: (1) easy incorporation of informatics within mainstream curricula, (2) centralization and standardization of curricular material, (3) editorial functionality for revisions and updates, (4) wide accessibility of material irrespective of venue, (5) search functionality for faculty and students, (6) the ability to perform curriculum inventory, and (7) the potential for use to compensate for decreased faculty time. The ongoing experience at McGill has shown that the merging of technology and pedagogy requires a substantial commitment of resources and recognition of faculty time and change-management issues.